Content caching is used by many networks and service provider to optimize the delivery of digital content. Copies of the same content are cached by different servers distributed throughout the network. When a request for the cached content comes in, the request can be routed to the optimal server that serves the requested content in the most expedient manner. The optimal server can be determined based various criteria including geographic proximity to the user, load, available resources, etc.
However, all caching servers have a finite or limited amount of storage that can be used as cache. When the cache of a particular caching server is filled, additional content may be cached by evicting other content from the cache. The caching servers are configured with caching methodologies to manage how and what content is stored to cache and eviction methodologies to manage how and what content is evicted from cache in order to free space for new content to be cached.
Some common caching methodologies implement first-hit or n-hit caching, whereby content is cached after the caching server receives one or n requests for the content. These caching policies are implemented to be lightweight so as to be able to handle high demand without creating a bottleneck.
Least recently used (LRU) and least frequently used (LFU) are two common eviction methodologies. Under LRU, content is evicted from cache according to recency of requests. Under LFU, content is evicted from cache according to frequency of requests.
From a holistic perspective, these caching and eviction methodologies appear to adequately manage the cache storage while being lightweight so as to not create a bottleneck during times of high demand. However, closer analysis reveals various inefficiencies with these prior art methodologies. Under these methodologies, a few content providers can disproportionately consume the cache at the expense of other content providers. Consequently, only a few content providers benefit from the efficiencies afforded by the cache, while other content providers receive little or no such benefit. These inefficiencies primarily arise because the prior art caching and eviction methodologies treat content or content providers the same. As a result, a first content provider with ten times more content than a second content provider could have a tenfold greater cache footprint than the second content provider even when the content of the second content provider is served from the cache more often than the content of the first content provider. Similarly, a first content provider with large sized content could have a larger cache footprint than a second content provider with small sized content even when the small sized content is served from the cache more often than the large sized content.
Accordingly, there is a need to better manage how and what content is stored to cache and evicted from cache. There is a need for smarter caching and eviction methodologies that provide differentiated and configurable access to the cache.